Video viewing system and method

ABSTRACT

An outgoing video stream is produced in a plurality of video modes, including a live play mode in which an input task ( 20 ) is coupled to a decoding task ( 22 ), a pause mode in which the input task ( 20 ) is coupled to a recording task ( 24 ), and a time-shift mode in which the input task ( 20 ) is coupled to the recording task ( 24 ) and a replay task ( 26 ) is coupled to the decoding task ( 22 ). In different modes, different subsets of the tasks ( 20, 22, 24, 26 ) or all of the tasks ( 20, 22, 24, 26 ) are active. FIFO communication buffers ( 16   a - d ) are used to communicate between the tasks ( 20, 22, 24, 26 ). Switching between the video modes is realized by reassigning connections of tasks ( 20, 22, 24, 26 ) to respective ones of the communication buffers ( 16   a - d ), keeping previous data in the reassigned FIFO communication buffers ( 16   a - d ). Tasks ( 20, 22, 24, 26 ) that are needed before and after a switch are kept continuously active, so that they continue to read from or write to the FIFO communication buffers ( 16   a - d ) to which they are connected. Reassignment of connections to communication buffers ( 16   a - d ) is preferably limited to points of time at transitions between transfer of closed Groups Of Pictures in die video streams.

The invention relates to a video viewing system which supportstime-shift viewing of a video stream.

PCT patent application WO99/33265 discloses a video viewing system whichis capable of operating in a number of modes of operation, including atime shift mode, a live play mode and a pause mode. In the time-shiftmode, an incoming video stream is recorded in a storage unit and inparallel therewith a significantly older (e.g. ten minutes older) partof the incoming stream is replayed to the output of the system from thestorage unit. In the live play mode, the incoming stream is fed directlyto the output without any significant delay. In the pause mode, thesystem outputs a still image, which is the last image output duringpreceding live play and simultaneously records the incoming video streamin the storage unit.

Video mode switching between the different modes is not described inWO99/33265. In principle, video mode switching could be achieved byrestarting the video system each time when a video mode switch commandis received, starting up data flows as required for the relevant mode.This, however, has the disadvantage that an interruption may occur inthe video output signal, as a result of the need to fill the variouspipe-lines in the system and to restart the decoding process, oftenstarting from an arbitrary point of time in the incoming video streamwhich does not permit decoding to start immediately.

An alternative is to realize video mode switches by means ofmultiplexing. In this case, the video system keeps all tasks that needto be executed in any one of the modes (i.e. at least an input task, arecording task, a replay task and a decoding task) running in all modes.One or more multiplexers switch the sources of one or more of the tasksduring a video mode switch. Because none of the tasks needs to berestarted, this potentially eliminates interruptions. However, it hasthe disadvantage that certain tasks keep running also when they are notneeded. In a video system, in which the tasks are implemented assoftware tasks running on a general-purpose processor, the processorresources occupied by the tasks remain unavailable to other processes.In a hardware-implemented system, the hardware that performs the taskscontinues to consume unnecessary power supply current.

European patent application No. 01203905.3 (unpublished at the prioritydate of the present application and assigned to the same assignee)describes a mechanism for dynamically reconfiguring networks ofprocesses or tasks.

It is, inter alia, an object of the invention to provide a video displaysystem which is capable of operating in several modes of operationincluding a time-shift mode, wherein video mode switching can berealized without significant interruption and a task not needed during acertain mode need not continue execution in that certain mode.

The video display system according to the invention is set forth inclaim 1. FIFO (First In First Out) communication buffers withreassignable sources and destinations are used for communicating videostreams between the input, decoding, recording and replay tasks. Duringa mode switch, dynamic reconfiguration is realized by reassigning thesource or destination of a FIFO communication buffer. The other side ofthe FIFO communication buffer stays assigned to the same task before andafter the mode switch. At the reassignment, video data may still bepresent in the FIFO communication buffer. The task that stays assignedto the FIFO communication buffer keeps on executing during the modeswitch, as far as video data is available in the FIFO communicationbuffer. Thus, the use of FIFO communication buffers which remain activeduring the mode switch ensures that mode switching can be performedwithout interruption of the video streams and without requiring tasks toremain active if they are no longer needed after the mode switch. Theresources used by such a task may be freed after the mode switch.

In an embodiment, dynamic reassignment involves detachment andsubsequent attachment of respective sources or destinations of videostreams to the communication buffer. Detachment and attachment occur inresponse to receiving a mode-switch command, but with delays until asubsequent boundary between closed Groups of Pictures has been detectedin the detached and attached stream, respectively. A closed Group ofPictures in a video stream encodes a series of frames of videoinformation that can be decoded without requiring access to framesoutside the group. By delaying detachment and attachment, it isprevented that “orphaned” video data occurs in the communication buffer,which cannot be decoded and might give rise to interruptions in thedecoding task.

These and other objects and advantageous aspects will be described withreference to the following Figures.

FIG. 1 shows a video display system,

FIGS. 2 a-c show task graphs during different video modes.

FIG. 1 shows a video display system. The system contains a video streaminput 10, a video stream output 12, a plurality of processing taskelements 14 a-f, a plurality of FIFO communication buffers 16 a-d, abuffer connection element 17, a storage device 18 and a switchingcontrol unit 15. A video display device 19 is coupled to output 12.Processing task elements 14 a-f include an input task element 14 a, arecording task element 14 b, a replay task element 14 c, a decoder taskelement 14 f plus optional further elements 14 d,e. Input 10 is coupledto input task element 14 a. Output 12 is coupled to decoder task element14 f. Storage device 18 is coupled to recording task element 14 b andreplay task element 14 c. Processing task elements 14 a-f are coupled toFIFO communication buffers 16 a-d via buffer connection element 17.Switching control unit 15 is coupled to processing task elements 14 a-f,FIFO communication buffers 16 a-d and buffer connection element 17.

In a first embodiment, each component shown in FIG. 1 is a dedicatedhardware element. In other embodiments, one or more of the componentsmay be implemented in a general purpose processor by means of suitablecomputer programs. For example, in the first embodiment, processing taskelements 14 a-f are each different hardware elements with a designdedicated to the task to be performed by the processing task element 14a-f but in the other embodiments part or all of the processing taskelements 14 a-f may be implemented as different computer programs loadedin a general purpose processor. Similarly, in the first embodiment, FIFOcommunication buffers 16 a-d are hardware FIFO buffers with data inputs,data outputs, and buffer full and empty signaling outputs. However, inthe further embodiments, FIFO communication buffers 16 a-d may beimplemented as different buffer areas in a processor memory withappropriate software for signaling to task elements 14 a-f whether ornot the buffers are full or whether or not they are empty. Alsosimilarly, in the first embodiment, buffer connection element 17 may bea hardware element such as a switch matrix and in the furtherembodiments buffer connection element 17 may be implemented as aninterface program between FIFO communication buffers 16 a-d andprocessing task elements 14 a-f. Furthermore, switching control unit 15may be implemented in hardware or as a software programmed computer.

In operation, an incoming video stream is applied to input 10 and anoutgoing video stream is produced at output 12, for display at videodisplay device 19. In between, processing task elements perform variousprocessing tasks on video data that derives from the incoming stream.

Video mode control unit 15 controls the video mode in which the videoviewing system operates. The combination of tasks executed depends onthe video mode in which the system operates. The video modes include alive play mode, a pause mode and a time-shift mode.

FIGS. 2 a-c show task graphs of the processing in different video modes.FIG. 2 a shows operation in the live play mode, in an embodiment wherethe incoming stream is an analog video signal. In this case, an inputtask 20 is performed which receives the incoming stream and outputs anencoded stream to a decoding task 22 via a first connection 21. Decodingtask 22 outputs a decoded stream for use by video display device 19 (notshown). Although a single decoding task 22 has been shown, it will beappreciated that the decoding task may in fact be comprised of severaltasks, such as a demultiplexing task for demultiplexing a program froman MPEG transport stream, followed by a program decoding task operatingon the demultiplexed program. Similarly separate video and audio streamprocessing tasks may occur. For reasons of simplicity, only a singletask is shown.

FIG. 2 b shows operation in the pause mode. In this mode, a recordertask 24 has been added. Input task 20 continues to be performed andoutputs the encoded stream to recorder task 24 via first connection 21.Recorder task 24 records the encoded stream in storage device 18 (notshown). Decoder task 22 has been switched to a “freeze” state, in whichit permanently outputs its last received video frame, without readingfrom its input. Its input connection has been uncoupled. In analternative embodiment, in which decoder task 22 is arranged topermanently output its last full received frame when it finds that nonew frame has been supplied to its input, first connection 21 may beleft connected to decoder task 22, a second connection being addedbetween input task 20 and recording task 24. In this alternativeembodiment, first connection 21 may be disconnected from input task 20.

FIG. 2 c shows operation in the time-shift mode. In this mode, the inputtask 20, the decoder task 22 and the recorder task 24 continue to beperformed and a replay task 26 has been added. Input task 20 outputs theencoded stream to recorder task 24 via first connection 21. Replay task26 outputs a delayed version of the input stream from storage device 18(not shown) to decoder task 22 via a second connection 23. Decoder task22 has been switched back to the normal operating state, in which itreads video data from its input and uses that data to update its output.

In the apparatus of FIG. 1, tasks 20, 22, 24, 26 are executed byprocessing task elements 14 a-c,f. Connections 21, 23 are implemented byFIFO communication buffers 16 a-d. Tasks that are not required in avideo mode are deactivated, for example, by disabling the clock input tothe processing task element in the case of hardware-implemented taskelements, or by releasing resources used by these tasks in the case ofsoftware-implemented task elements, thus releasing system resources forother uses.

When video mode control unit 15 switches the video display systembetween different video modes, the processing task elements 14 a-c,fthat are active before and after the mode switch are kept active (or atmost suspended) through the mode switch, so that they will continueexecution after the mode switch. The FIFO communication buffers 16 a-dcoupled to these continuously active processing task elements 14 a-c,falso remain active, but video mode control unit 15 controls bufferconnection element 17 so that the coupling to or from some of the inputsor outputs of FIFO communication buffers 16 a-d are switched todifferent processing task elements 14 a-c,f.

When a first one of processing task elements 14 a-c,f has an input oroutput coupled, via a FIFO communication buffer 16 a-d, to a second oneof processing task elements 14 a-c,f before the mode switch and to athird one of processing task elements 14 a-c,f after the mode switch,the second one of the processing task elements 14 a-c,f is decoupledfrom this FIFO communication buffer 16 a-d and the third one is coupledto this FIFO communication buffer 16 a-d in its place during the videomode switch. Video data in the FIFO communication buffer 16 a-d is leftin place for later use. The second one of processing task elements 14a-c,f is deactivated (or its execution suspended) before being decoupledand the third one is activated or unsuspended after being coupled (or,if activated earlier, it is supplied with a dummy “FIFO full” signal ifits output is coupled to the FIFO communication buffer 16 a-d or “FIFOempty” signal if its input is to be coupled). Thus, no data is lostduring mode switching and there is no interruption of the video signals.

In the case of a switch from the live play mode of FIG. 2 a to the pausemode of FIG. 2 b, video mode switch control unit 15 sends a signal tothe decoding task element 14 f to switch to the “freeze” state ofoperation, causing it to repeat a current frame. Video mode switchcontrol then disconnects the FIFO communication buffer 16 a-d thatimplements connection 21 from decoding task element 14 f and connects itto a recording task element 14 b. Subsequently, video mode switchcontrol unit 15 activates the recording task element 14 b.

In the alternative embodiment, in which first connection 21 remainscoupled to decoder task 22, video mode switch control unit 15 reconnectsthe output of input task element 14 a to a second FIFO communicationbuffer 16 a-d (optionally causing input task 20 to suspend operationduring reconnection) and connects recording task element 14 b to thatsecond FIFO communication buffer 16 a-d before activating recording taskelement 14 b. In this embodiment, decoder task 22 does not have to bebrought into a freeze state. This simplifies design, but it has thedisadvantage that some delay may occur during video mode switchingbecause buffered data from first connection 21 has to be processedbefore decoder task 22 outputs a frozen image.

In the case of a switch from the pause mode of FIG. 2 b to thetime-shift mode of FIG. 2 c, video mode switch control unit 15 connectsa replay task element 14 c and the decoding task element 14 f to asecond FIFO communication buffer 16 a-d that implements connection 23.Subsequently, video mode switch control unit 15 signals decoding taskelement 14 f to return to the normal state of operation, in which itreads video data for new frames from second FIFO communication buffer 16a-d, and activates the replay task element 14 b. In the alternativeembodiment, wherein the first connection 21 remains attached to decodingtask 22, no new connection needs to be added of course. In this case,video mode switch control unit 15 couples replay task element 14 c tothe FIFO communication buffer 16 a-d that implements first connection 21and activates replay task element 14 c. No signal is needed to switchdecoder task element 14 f back to the normal operating state in thiscase.

In the case of a switch back from the time shift mode of FIG. 2 c to thelive play mode of FIG. 2 a, video mode switch control unit 15 signalsreplay task element 14 c to cease execution. Video mode switch controlunit 15 commands recording task element 14 b and replay task element 14c to cease execution. Then video mode switch control unit 15 reconnectsthe input of decoding task element 14 f from the FIFO communicationbuffer 16 a-d that is connected to replay task element 14 c to the FIFOcommunication buffer 16 a-d that implements first connection 21 to inputtask element 14 a. Preferably, a signal is sent to switch decoder taskelement 14 f to the “freeze” mode temporarily during reconnection.Afterwards, the FIFO communication buffer 16 a-d implementing connection23 may also be released and its contents discarded.

Alternatively, input task element 14 a may be coupled to the input ofthe FIFO communication buffer 16 a-d that couples replay task element 14c to decoder task element 14 f in the time shift mode. However, this hasthe disadvantage that switching from time shift replay to live playoccurs only with some delay, because decoder task element 14 f has toread the old video data from the FIFO communication buffer 16 a-d first.

In the alternative embodiment, wherein decoder task 22 remains coupledto first connection 21, video mode switch unit 15 reconnects the inputof decoder task 22 to the FIFO communication buffer 16 a-d thatimplements second connection 23. It will be seen that thus, throughswitching from the live play mode to the pause mode, to the time shiftmode and back to the live play mode, a different one of FIFOcommunication buffers 16 a-d is coupled to encoder task element 14 fcompared with the initial live play mode. This is a result of dynamicassignment of the FIFO communication buffers 16 a-d.

It will be appreciated that in this way no restart of any processingtask elements 14 a-f is needed when the element is active both beforeand after the video mode switch. Processing task elements 14 a-f thatare no longer needed are deactivated, releasing system resources forother use. Continued use of FIFO communication buffers 16 a-d before andafter the video mode switch ensures continuity of the viewed videostreams. It will also be appreciated that a similar implementation canbe used for the reverse of the switches that have been described (i.e.switches from time shift mode to pause mode, from pause mode to liveplay mode and from live play mode to time shift mode).

Preferably, the processing task elements 14 a-f are designed to selectthe point of time where they cease to be active dependent on theprogress of the video data being processed, so that the part of thestream after ceasing can be decoded without reference to earlier videodata. In an MPEG video signal, for example, I-frames, P-frames andB-frames occur, the latter two types of frame being encoded as updatesto other frames. Closed “Groups Of Pictures” (GOPs) are identified inthe video stream, so that the frames in each particular GOP are neverencoded as updates to frames outside that particular GOP.

In this case, when video mode control unit 15 signals to a processingtask element 14 a-f to suspend or cease operation as part of a videomode switch, the processing task element 14 a-f finishes reading and/orwriting of a GOP from or to a FIFO communication buffer 16 a-d prior toactually ceasing or suspending operation. Thereupon the processing taskelement 14 a-f signals back to video mode switch control unit 15 thatthe command has been executed. Subsequently, video mode switch controlunit 15 signals buffer connection element 17 to couple the buffer orbuffers involved to another processing task element 14 a-f as requiredby the mode switch. After that, video mode control unit signals thenewly connected processing task element 14 a-f to start or resumeexecution.

FIFO communication buffers 16 a-d preferably provide video data input,video data output and empty/full signaling. The data width is notrelevant to the invention but typically multibit datawords will bewritten and read. Each FIFO communication buffer 16 a-d provides afull/not full signal back to the processing task element 14 a-f thatsupplies data to the buffer. If “full” is signaled and the processingtask element 14 a-f has data available, the processing task element 14a-f suspends execution. Each FIFO communication buffer 16 a-d providesan empty/not empty signal to the processing task element 14 a-f thatreads data from the buffer. If “empty” is signaled and the processingtask element 14 a-f needs to read data, the processing task element 14a-f suspends execution. In the software implementation of processingtask element 14 a-f, the processing task element 14 a-f may use pollingof the empty and full signals to determine when to resume operation, orevent signaling or interrupt signaling may be used to resume operation.In the hardware implementation, the empty and full signals may also beused as enable or disable signals.

FIFO communication buffers 16 a-d may be implemented as memory areas ina computer memory, the FIFO operation being controlled by software(keeping a pointer to the oldest unread data and the free location nextto last written data, checking for full and empty conditions and settingflags accordingly or signaling those flags to coupled processing taskelements 14 a-f). Dedicated hardware FIFO buffers may be used as well.Similarly, buffer connection element 17 may be implemented usingsoftware, in which case buffer connection element 17 may be integratedwith FIFO communication buffers 16 a-d in the form of pointers toprocessing task elements 14 a-f to which respective FIFO communicationbuffers 16 a-d are coupled. Alternatively, buffer connection element 17may be integrated with processing task elements 14 a-f in the form ofpointers to FIFO communication buffers 16 a-d to which respectiveprocessing task elements 14 a-f are coupled. Mixtures of such pointerimplementations are also possible.

It will be appreciated that the invention is not limited to the specificembodiments that have been described.

1. A video viewing system, comprising: an input (10) for receiving anincoming video stream; an output (12) for outputting an outgoing videostream; a storage device (18) for storing video data; a plurality ofFIFO communication buffers (16 a-d); a processing system (14 a-f, 15)arranged to execute a plurality of tasks, including an input task (20)coupled to the input, a decoding task (22) coupled to the output, arecording task (24) for recording on the storage device (18) and areplay task (26) for replaying from the storage device (18), theprocessing system (14 a-f, 15) being switchable between a plurality ofvideo modes, including a live play mode in which the input task (20) iscoupled to the decoding task (22), a pause mode in which the input task(20) is coupled to the recording task (24), and a time-shift mode inwhich the input task (20) is coupled to the recording task (24) and thereplay task (26) is coupled to the decoding task (20), the tasks beingcoupled via respective ones of the FIFO communication buffers (16 a-d),the processing system (14 a-f, 15) being arranged to switch between thevideo modes by reassigning connections of tasks (20, 22, 24, 26) to therespective ones of the FIFO communication buffers (16 a-d), keepingprevious video data in the reassigned FIFO communication buffers (16a-d), keeping tasks (20, 22, 24, 26) that are needed before and after aswitch continuously active and ceasing execution of tasks that are notin use after the switch.
 2. A video viewing system as claimed in claim1, wherein the processing system (14 a-f, 15) is arranged to send a“freeze” signal to the decoder task (22) upon a mode switch from thelive play mode to the pause mode, to make the decoder task (22) enter afreeze state in which the decoder task (22) permanently outputs acurrent video frame.
 3. A video viewing system as claimed in claim 2,wherein the input task (20) is coupled to the decoder task (22) via afirst one of the FIFO communication buffers (16 a-d) in the live playmode, the processing system (14 a-f, 15) being arranged to decouple aninput of the decoder task (22) from an output of the first one of theFIFO communication buffers (16 a-d) upon switching from the live playmode to the pause mode, to couple an input of the recording task (24) tothe output of the first one of the FIFO communication buffers (16 a-d)and subsequently to activate the recording task (24).
 4. A video viewingsystem as claimed in claim 3, wherein the processing system (14 a-f, 15)is arranged to couple an output of the replay task (26) to an input of asecond one of the FIFO communication buffers (16 a-d) upon a video modeswitch from the pause mode to the time-shift mode, to couple an input ofthe decoder task (22) to an output of the second one of the FIFOcommunication buffers (16 a-d) and subsequently to activate the replaytask (26), the decoder task (22) being switched back from the freezestate to a normal operating state.
 5. A video viewing system as claimedin claim 1, wherein the input task (20) is coupled to the decoder task(22) via a first one of the FIFO communication buffers (16 a-d) in thelive play mode, the processing system (14 a-f, 15) being arranged tocouple a second one of the FIFO communication buffers (16 a-d) betweenthe input task (20) and the recording task (24) upon switching from thelive play mode to the pause mode, an input of the first one of the FIFOcommunication buffers (16 a-d) being disconnected from the input task(20) and an output of the first one of the FIFO communication buffers(16 a-d) remaining coupled to an input of the decoder task (22).
 6. Avideo viewing system as claimed in claim 1, wherein the input task (20)is coupled to the recording task (22) in the time shift mode via a firstone of the FIFO communication buffers (16 a-d), the processing system(14 a-f, 15) being arranged to reconnect an input of the decoder task(22) to an output of the first one of the FIFO communication buffers (16a-d) upon a video mode switch from the time shift mode to the live playmode, deactivating the replay task (26) and the recording task (24). 7.A video viewing system as claimed in claim 1, wherein the input task(20) is coupled to the recording task (24) in the time shift mode via afirst one of the FIFO communication buffers (16 a-d), and the replaytask is coupled to the decoder task via a second one of the FIFOcommunication buffers, the processing system (14 a-f, 15) being arrangedto deactivate the replay task (26), to deactivate the recording task(24), to couple a second one of the FIFO communication buffers (16 a-d)to the input task and to release the first one of the FIFO communicationbuffers (16 a-d) upon a switch from the time shift mode to the live playmode.
 8. A video viewing system as claimed in claim 1, wherein each task(20, 22, 24, 26) that is deactivated upon mode switching is arranged todelay deactivation upon mode switching until it has processed a closedgroup of pictures and written or read that group to or from one of theFIFO communication buffers (16 a-d).
 9. A video viewing system asclaimed in claim 1, wherein at least one of the replay task (26) andrecording task (24) is implemented as a computer program running on aprogrammable computer, so that resources used by said at least one ofthe replay task (26) and recording task (24) are released in a videomode or modes when said at least one of the replay task (26) andrecording task (24) is not active.
 10. A video viewing system as claimedin claim 1, wherein at least one of the replay task (26) and recordingtask (24) is implemented in dedicated hardware, the dedicated hardwarebeing switchable to a power-saving state, the dedicated hardware forsaid at least one of the replay task (26) and recording task (24) beingreleased in a video mode or modes when said at least one of the replaytask (26) and recording task (24) is not active.
 11. A method ofproducing an outgoing video stream for viewing, wherein a plurality oftasks (20, 22, 24, 26) is executed, including an input task (20)receiving an incoming video stream, a decoding task (22) outputting theoutgoing video stream, a recording task (24) recording on a storagedevice (18) and a replay task (26) replaying from the storage device(18), the method comprising the steps of: switching between a pluralityof video modes, different subsets of the tasks (20, 22, 24, 26) or allof the tasks (20, 22, 24, 26) being active in different ones of themodes, the video modes including a live play mode in which the inputtask (20) is coupled to the decoding task (22), a pause mode in whichthe input task (20) is coupled to the recording task (24), and atime-shift mode in which the input task (20) is coupled to the recordingtask (24) and the replay task (26) is coupled to the decoding task (22),using FIFO communication buffers (16 a-d) to communicate between thetasks (20, 22, 24, 26), switching between the video modes being realizedby reassigning connections of tasks (20, 22, 24, 26) to respective onesof the communication buffers (16 a-d), keeping previous data in thereassigned FIFO communication buffers (16 a-d), keeping tasks (20, 22,24, 26) that are needed before and after a switch continuously activeand ceasing execution of tasks (20, 22, 24, 26) that are not in useafter the switch.
 12. A computer program product comprising a program ofcomputer instructions for making a programmable computer perform themethod of claim 11.